Interactive teaching and learning device

ABSTRACT

The invention relates to a device which allows to preferably explain and demonstrate three-dimensional objects, such as anatomic models or also models and exhibits for museums and fairs. According to the invention the model  1  is fastened to the adjacencies by at least one multiple-component force-torque measurement device  2 , includes an electronic storage and evaluation unit and an optic-visual and/or acoustic indicating device. The force-torque measurement device  2  converts the forces and moments arising when the model  1  is touched into electrical measurement signals to be leaded to the electronic storage and evaluation unit, and in the electronic storage and evaluation unit the contact zone is calculated from the forces and torques detected as a result of the touch, and is communicated to the operator as a signal by means of the optic-visual and/or acoustic indicating device.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of prior filed copending U.S.application Ser. No. 10/541,295, filed Jul. 18, 2005, the priority ofwhich is hereby claimed under 35 U.S.C. §120 and which is the U.S.National Stage of PCT International Application No. PCT/DE03/04292,filed Dec. 31, 2003, which designated the United States and has beenpublished but not in English as International Publication No. WO2004/061797 and which claims the priority of German Patent Application,Serial No. 102 61 673.6, filed Dec. 31, 2002, pursuant to 35 U.S.C.119(a)-(d).

The contents of U.S. application Ser. No. 10/541,295, InternationalApplication No. PCT/DE03/04292, and German Patent Application, SerialNo. 102 61 673.6 are incorporated herein by reference in its entirety asif fully set forth herein.

BACKGROUND OF THE INVENTION

The invention relates to a device which allows to preferably explain anddemonstrate three-dimensional objects, e.g. anatomical models or evenmodels and exhibits for museums and fairs.

In the field of medical training or in medical demonstrations anatomicalmodels of plastic material or other materials are frequently used. Forthe explanation or accentuation of certain anatomical areas it is oftenadvisable to mark the relative areas by inscriptions or coloured signs.

The problem with regard to such models is that for the reason of lack ofspace the information to be imparted by means of an inscription must notbe very voluminous. In many cases the inscription is completely omitted,as, for example, the texture of the model (colouring, fine vessels,nerves etc., are to remain recognizable. Therefore the names andinformative details belonging to the individual areas of a model arelisted on a sheet of paper. The assignment follows from numbersindicated on the model, or from sketches or photos which show therelative areas of the model. Therefore the identification of the modelareas of interest is often very complicated and unclear.

The same problems apply to the construction of three-dimensionaldemonstration models shown in museums or at fairs, in whichcases—contrary to medical models—even original objects, such as anoldtimer vehicle in an automobile museum, may be concerned.

Regarding these museum and fair models it may also be advisable to makeinscriptions or coloured marks for illustrating, describing oraccentuating certain areas or elements of the model. For this purposeelectrical switches are often used which—after being touched on themodel or away from it—ensure that a certain area of the model becomesvisible by means of small incandescent lamps or is explained by means ofan inscription lighted up. So-called touchpads are used for specialapplications allowing the detection of a flat force distribution on thebasis of sensing elements arranged in a matrix array, please see also DE36 42 088 C2. The disadvantage of such arrangements is that there aresensor components between the touched model and the operator so that theoriginal contact properties, such as surface condition, shape andcolour, are distorted. Furthermore, owing to the mounting of the sensorcomponents the model to be touched has to be processed. As a result themodel may be changed or even damaged. Further, in order to achieve asufficient level of space resolution over the whole of the concernedmodel area, a plurality of sensors sensitive to pressure have to beused.

These disadvantages are partly removed by the usage of so-callednavigation or tracking systems which detect the contact point not on theside of the model but on the side of the operator, e.g. by tracking theoperator's finger or instrument. The range of equipment required for thedetection of the operator's movement, however, is excessive.

SUMMARY OF THE INVENTION

Therefore it is the object of the invention to provide improved modelsfor learning and demonstrating purposes which, above all, overcome theabove mentioned disadvantages.

This task is solved by a device according to claims 1 and 2:

According to claim 1 a teaching and learning device showing thefollowing characteristics is provided: A 3D body incorporating the modelis fastened to the adjacencies by at least one multi-componentelectrical force-torque measurement device. When the 3D body is touchedthe forces and torques arising are converted into electrical measurementsignals which are leaded to an electronic storage and evaluation system.In the electronic storage and evaluation system a mathematical model ofthe geometry of the 3D body is implemented. Hereinafter geometry meansat least each surface area of the model which can be contacted and whichis to be explained, i.e. also body hollows of an anatomical model.

Furthermore an algorithm known as such from the state of the art isimplemented, which calculates the place at the 3D body just beingtouched, for example by a finger or a needle, from the forces andtorques detected as the result of the contact.

The calculated place of the contact is indicated or displayed by meansof an indicating device. The mode of indication and/or output isoptional and is executed in accordance with the purpose to be achieved.Optic-visual and/or acoustic indicating devices as known from the stateof the art are preferred.

The invention according to claim 2 as an invention on its own issubordinated to the same basic idea as the invention according to claim1.

The fundamental difference, however, is that no mathematical model isstored in the electronic storage and evaluation system, but a data tablein which the contact points of interest are stored.

These contact points are implemented by means of the “teaching” methodknown from the state of the art, which means that the place to be“taught” on or in the 3D body (for example a body hollow) is touched bya finger or an instrument, thereby applying a predetermined force whichis transferred to the multiple-component force-torque measurementdevice.

The forces and torques detected by the multiple-component force-torquemeasurement device are compared with the data stored in the data table.By means of an assignment algorithm the place touched is detected anddisplayed by the indicating device. Contrary to the invention accordingto claim 1, which, on principle detects any point as far as it iscovered by the mathematical model, the invention in accordance withclaim 2 can practically detect the pre-taught points only.

The model is fastened to a table, a wall, a ceiling or any other basesurface by only one multiple-component force-torque measurement device.For the reason of a better mechanical stability even several forcemeasurement devices may be used. Multiple-component force-torquemeasurement devices are part of the state of the art and arecommercially offered as modular components. Additional holdingappliances may also be used if required by the dimensions of the 3Dbody. These holding appliances, however, must be constructed in a way tounambiguously and reproducibly feed the force caused by the touch to theforce-torque measurement device or the force-torque measurement devices.

The outstanding feature compared with the devices hitherto known is thatthe touch-sensitive sensor system is not positioned at the touch pointof the model but is arranged as connecting element between the model andthe adjacencies. For this reason there is no need to expensively adaptthe model. Furthermore nearly any number of touch points may begenerated, which is not possible with regard to the devices of the knownart.

The construction as mentioned above allows to visually and/oracoustically explain, describe or accentuate the areas, points ofelements of the model touched by the operator. For example the detailsshown may be the name, certain properties and functions of the area orelement of the identified model. The details are made readable orvisually recognizable by means of a visual display unit, and/or audibleby means of loudspeakers. Also films or graphic animations can beimported depending on what kind of setting and operating activities havebeen made. Further the amount of the force detected and the direction ofthe force detected can be further processed by the data processor andreproduced as a graphically animated vector arrow or as an acousticsignal. If for example, the operator applies too high forces to themodel a visual or acoustic warning signal or a warning voice may ensurethat the operator stops applying force to the model so as to avoid adestruction of the model or the force sensor.

The mathematical representation of the used model can be determined bymeans of 3D-scanners (CT, magnetic resonance tomography, laser scanneretc.) and stored in a data processor. When the teaching method is usedthe relative areas of the model are touched, and the thereby arisingforces and torques are measured and stored and assigned, for example bythe input of texts. In this case the assignment method can be supportedby up-to-date techniques such as artificial neural networks. As soon asin the course of the later application forces arise which are comparablewith those measured in the teaching process, the element touched isdetected automatically.

The geometric image of the model can also be represented in agraphically animated way. In the animation certain areas of the modelwhich are touched can be marked by colour or by means of an arrow. Evenvery fine details which are positioned near the touch point but cannotbe marked on the real model for lack of space can be visualized by meansof the visual display unit.

On the model or within certain predetermined areas of the model severaldistinguishable menu points which optically differ in colour, size,shape, inscription can be marked. If one of these menu points istouched, depending on the kind of the point a certain reaction isreleased or menu function is executed which is displayed acoustically orgraphically.

Alternatively or in addition to the points which are opticallydistinguishable, certain touch patterns with typical force/timebehaviours may lead to various graphic and acoustic responses. Suchtouch patters are for example: long or short contacts, light or strongcontact pressing, as well as tapping signs with varying numbers of tapssuch as the double click in the Windows programme which leads to theopening of a file.

The invention can be operated in two different modes. The abovementioned function represents the so-called standard mode, in which thetouch results in a graphic and/or acoustic response. In the so-calledinquiry mode at first a graphic or acoustic request can be put to theoperator such as to touch a certain area of the model. Thereupon theoperator, e.g. a student to be examined, touches the supposed area, andthe data processor checks whether the correct area has been touched,i.e. detected. As a result it is further possible to verify whether theoperator has contacted the areas in the right order and, if required,also in the correct periods of time and by applying the correct amountsand directions of forces. Success, failure or a valuation are thencommunicated to the operator by means of the graphic and/or acousticdisplay. By using this mode the operator's knowledge is tested.

According to claim 3 the optic-visual indicating device includes aprojector which projects visual data such as texts or images directly tothe area touched, which also allows to project the reverse sides. It isrequired, however, that the colour and the surface of the model area areadjusted to match the projection. If, for example, the operator withgrowing force presses the lung of the model, more low-lying sections areprojected and represented. It is known to the specialist that suchprojections can be shown on separate monitors as well.

According to claim 4 the projector is provided as video projector. This,for example, allows to show the blood transportation in the lung in away very similar to reality, thus further improving the informativeeffect.

Further it is to be mentioned that there is a number of intelligentalgorithms for the evaluation of the signals of the force-torquemeasurement device. In case of a dismountable anatomic model, forexample, the remaining mass is reduced when an organ is removed.Therefore, if the masses of the dismountable organs are different andknown, it is possible to determine the dismounted organ by a simpleweight classification. It is further possible to utilize the shifting ofthe centre of gravity of the model on removal of an organ for thedetermination. If a certain organ is removed, on principle theforce-torque measurement device does not only record a reduction inweight but also a tilting moment. To minimize the possibility ofconfusion it is further possible to provide for algorithms forplausibility checks. Consequently, if, for example, two organs are ofthe same weight, however, are positioned one behind the other andtherefore can be removed only in the predetermined order, the organ justremoved can be clearly identified.

Now the description of the invention will be made at greater detail bymeans of examples of embodiments and schematic drawings:

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 a-f show the application of the invention to a model of ananatomic torso.

FIG. 2 shows the application of the invention to a model ear for thetraining in acupuncture.

FIG. 3 shows an embodiment with divided model.

FIG. 4 a, b show an embodiment of the invention for a non-medicalapplication.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 a shows an artificial open upper part of a body 1 (phantom torso)with dismountable organs. In this embodiment the invention serves tosupport the medical training. The torso is mounted on a 6-componentforce-torque sensor 2. The sensor data lead to a data processing unitwith graphic and acoustic output. On the individual organs there areseveral small dots in yellow, blue and green colour. If, for example, astudent of medicine touches one of the organs or a certain area of anorgan, the name of the relative organ or area is communicated to himacoustically. Simultaneously a monitor shows the torso as artificialimage in a shaded way and the name of the area touched is inserted. Bythe way of graphic animation the touched structures can be accentuatedin colour. Even very fine anatomic structures, such as blood vessels,veinlets, lines of nerves, base points of muscles, can be made visible.If then an operator touches the yellow dot on the artificial organ ofthe torso, a photorealistic view of the organ or the area of the organis represented to him on the monitor. In case of the blue dot thephysiological relevance and possible pathologies are graphically andacoustically described. After all the green dot allows to start thegraphic animation and films with sound. Further by an increase inpressure on an organ or the skin of the torso model it becomes possibleto dip into the depth like a pin prick. As a result various bodysections and internal sights are graphically represented in an animatedway. In the inquiry mode (control mode) an artificial voice can requestthe operator to touch a certain area which is relevant from the anatomicpoint of view. The place touched is then recorded by the data processingunit and the result is acoustically and graphically communicated andcommented to the operator.

FIG. 1 b shows the operator removing one of the organs from the torso.As a result the sensor records an amended weight and a shifting of thecentre of gravity. As the weights of the individual components areknown, the sensor automatically detects the organ which has beenremoved. Thereupon the artificial display of the torso on the monitoradjusts itself according to the amended torso.

FIG. 1 c shows how after the removal of several parts of organs morelow-lying structures that have not been visible so far become visiblenow and can be explored further by touching them and by means ofacoustic-graphic support.

FIG. 1 d shows a different graphic and acoustic display using ahead-mounted-display (HMD). By the projection of two separate images toboth eyes a realistic three-dimensional image impression is achieved.The acoustic message is communicated to the operator by means of stereoheadphones.

FIG. 1 e shows a different graphic display in which the text and imagedata are projected directly to the touched model. This can be realizedby means of a commercial projection beamer, in which case as for thisexample the model surface is to be white or unicoloured in a lightcolour.

FIG. 1 f shows an embodiment in which the phantom torso is fastened bytwo multiple-component sensors 2 a, 2 b. The relative force-torquesignals are vectorially added up and finally further processed by thedata processing unit as sum signal which corresponds to the signal ofone single sensor.

FIG. 2 shows an embodiment in which a phantom ear is utilized for theacupuncture training. The phantom ear is connected with a force-torquesensor 2. The ear shows marks of the most important acupuncturepositions. If the operator by means of a sharp-pointed object which issimilar to an acupuncture pin touches the phantom ear, a voice and themonitor image tell him the name and the effect of the aimed dot. In thisexample of application the acoustic information and the text insertionsare meaningful also for the reason that there is not enough space on theear for the names and effects of the dots. Sound and image can alsoguide the operator when he looks for a desired dot. Further it is alsopossible to check how much time is taken by the operator to look for acertain dot and in which sequence he approaches these dots.

FIG. 3 shows an embodiment in which the model is divided. This meansthat the right model part is connected with the table by a force-torquesensor 2 a. The left model part, however, is connected with the rightmodel part by means of a further force-torque sensor 2 b. The sensor 2 bis the only connecting element between the right and the left modelparts. By this arrangement 2 forces—one per model part—can be initiatedand localized. This arrangement also allows ambidextrous pointingactivities. During the data processing the forces active at the leftpart can unambiguously be further processed by the connecting sensor 2b. As, however, the sensor 2 a on the side of the table receives theforces of both model parts, it is necessary for the localization of theright contact point that both sensor ends are coupled to each other. Forthis purpose the force-torque data of the connecting sensor arecomponent for component subtracted, i.e. vectorially, from theforce-torque data of the sensor on the side of the table (in a commoncoordinate system).

FIG. 4 a shows a model car mounted on a 6-component force-torque sensor2. The force-torque data are leaded to a data processing unit which hasan acoustic output facility by means of a sound generator (sound card).The data processing unit includes a mathematical image of the model cargeometry. The model cars is composed of a plurality of small components,such as wheels, doors, bumpers, headlights. As soon as the operator(visitor of a museum) shortly touches one of the components by hisfinger, he hears the name of the touched component by means ofloudspeakers. If he two times in a row quickly taps the same element,its function is explained to him in more detail. Simultaneously with theoutput of the acoustic information the monitor shows an animated imageof the model with a coloured accentuation of the touched part and a textbox which explains the function in more detail. One single long tappingstarts a short film which describes the manufacturing process of thetouched part.

FIG. 4 b shows an embodiment in which the model car is fastened by twomultiple components 2 a, 2 b. The relative force-torque signals arevectorially added and finally as a sum signal which corresponds to thesignal of a single sensor further processed by the data processing unit.

It is obvious that instead of the model car also a real object such asan automobile can be equipped with the invention. The particularsignificance in the fields of application: museum, exhibition or fair,without doubt, consists in the novel interaction of the exhibited objectwith the public that up to this time often has not been allowed to touchthe exhibits.

1. An interactive teaching and learning device which comprises a 3D body(1) to be touched which is fastened to the adjacencies by means of atleast one multiple-component force-torque measurement device (2), anelectronic storage and evaluation system, an optic-visual and/oracoustic indicating device, whereby the force-torque measurement deviceconverts the forces and moments arising when the model body is touchedinto electrical measurement signals to be leaded to the electronicstorage and evaluation unit, while a mathematical model of the geometryof the 3D body is implemented in the electronic storage and evaluationunit, and an algorithm which on the basis of the forces and torquesdetected when the touch is carried out calculates the contact zone atthe 3D body, which is communicated to the touching operator as signal bymeans of the optic-visual and/or acoustic indicating device.
 2. Ateaching and learning device which comprises a 3D body (1) to be touchedwhich is fastened to the adjacencies by at least one multiple-componentforce-torque measurement device (2), an electronic storage andevaluation unit, an optic-visual and/or acoustic indicating device,whereby the force-torque measurement device converts the forces andmoments arising when the model body is touched into electricalmeasurement signals to be leaded to the electronic storage andevaluation unit, force-torque measurement signals of predeterminedcontact points are stored in the memory of the electronic storage andevaluation unit, and an assignment algorithm is implemented which basedon the detected forces and torques assigns the contact zone at the 3Dbody which is communicated to the touching operator as signal by meansof the optic-visual and/or acoustic indicating device.
 3. A teaching andlearning device according to claim 1 characterized in that theoptic-visual indicating device comprises a projector projecting visualdata, such as texts or images, directly to the area touched.
 4. Ateaching and learning device according to claim 3 characterized in thatthe projector is a video projector.
 5. A teaching and learning deviceaccording to claim 2 characterized in that the optic-visual indicatingdevice comprises a projector projecting visual data, such as texts orimages, directly to the area touched.
 6. A teaching and learning deviceaccording to claim 5 characterized in that the projector is a videoprojector.